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DETAILED ACTION 



1 . This action is responsive to communications: Amendment filed 09/21/2006 to the 
original application filed 03/06/2000. 



Claims 1-13 are currently pending in this application. Claim 1 has been amended. Claim 
1 is an independent claim. 



Claim Rejections - 35 USC § 103 



2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A potent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, 
if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole 
would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter 
pertains. Patentability shall not be negatived by the manner in which the invention was made. 

This application currently names joint inventors. In considering patentability of the claims under 35 U.S.C. 103(a), the examiner 
presumes that the subject matter of the various claims was commonly owned at the time any inventions covered therein were made 
absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and 
invention dates of each claim that was not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e). (f) or (g) prior artunder 35 U.S.C. 103(a). 



Claims 1-13 are rejected under 35 U.S.C. 103(a) as being unpatentable over Yeo et al. 
(US 5,821,945, issued 10/1998) in view of Boetje et al. (US 6,049,332, filed 10/1996). 
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As to claim 1 : 

Yeo teaches a method for ordering multimedia content (see the Abstract)^ comprising the 
steps of: 

• segmenting the multimedia content to extract objects (coL2, lines 35-47); 

• extracting (coL4, lines 62'Col5, line 2) and associating features of the objects to 
produce content entities (col3, line 21'Col4, line 57)^ wherein the content entities 
are recursive data structures comprising features (col9, lines 42-56), relations 
(col, 4, lines 30-35 and col5, line 64-coL6, line 11), directed acyclic graphs (col 
3, line 30-col4, line 58); 

• coding the content entities to produce directed acyclic graphs of the content 
entities (col 5, line 30'Col4, line 10), each directed acycUc graph representing a 
particular interpretation of the multimedia content (col J, line 25- col5, line 52); 

• measuring attributes of each content entity (col6, lines 24-31); 

• assigning the measured attributes to each corresponding content entity in the 
directed acyclic graphs (col6Jine 51-col8, line 7) to order the content entities of 
the multimedia content (col 10, lines 19-32 & see figs. 4-5 and the associated 
text). 

Yeo, however, does not specifically teach ''containment sets and comparing the ordered 
content entities in a plurality of the directed acyclic graphs to determined similar 
interpretations of the multimedia content.'' 
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Boetje teaches containment sets (col 19, line J-coL 20, line 64; and see figs. 9-1 Ob and 
the associated text) and comparing the ordered content entities in a plurality of the 
directed acyclic graphs to determined similar interpretations of the multimedia content 
(coL 33, line 40-col. 34, line 62; and see figs, 22a'b and the associated text). 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to modify Yeo with Boetje because Boetje's teaching would have provided the 
capability for identifying both visual and temporal relationships to allow the user to 
recognize the underlying story structure and navigate to the desired point in the video. 

As to claim 2: 

Yeo teaches the measured attributes include intensity attributes (coL7, line 35-coL8, line 
8). 

As to claim 3: 

Yeo teaches the measure attributes include direction attributes (col.7, lines 13-19). 
As to claim 4: 

Yeo teaches the measured attributes include spatial attributes and the order is spatial 
(coL9, line 42-col.lO, line 7). 
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As to claim 5: 

Yeo teaches the measured attributes include temporal attributes and the order is temporal 
(col. 5, line 65-col 5, line 21 and col9, line 42-coLlO, line 7). 

As to claim 6: 

Yeo teaches the measured attributes are arranged in an increasing rank order (coL5, line 
65-coL6, line 11 and col. 10, lines 32). 

As to claim 7: 

Yeo teaches the measured attributes are arranged in an decreasing rank order (col. 5, line 
65-col. 6, line 11 and col. 10, lines 32). 

As to claim 8: 

Yeo teaches traversing the multimedia content according to the directed acyclic graph 
(col. 3, line 24- col. 4, line 10) and the measured attributes assigned to the content entities 
(col. 6, line 24-coL8, line 7). 

As to claim 9: 

Yeo teaches summarizing the multimedia content according to the directed acyclic graph 
(col.9, lines 19-42) and the measured attributes assigned to the content entities (col. 6, 
line 24-col.8, line 7). 



Application/Control Number: 09/518,937 



Art Unit: 2176 



Page 6 



As to claim 10: 

Yeo teaches the multimedia content is a three dimensional video sequence (col 7, lines 
12-19). 

As to claim 11: 

Yeo teaches nodes of the directed acyclic graphs represent the content entities and edges 
represent breaks in the segmentation, and the measured attributes are associated with the 
corresponding edges (col 5, lines 37-52), 

As to claim 12: 

Yeo teaches at least one secondary content entity is associated with a particular content 
entity, and wherein the secondary content entity is selected during the traversing (col2, 
lines 35-48 and col6, lines 1-21). 

As to claim 13: 

Yeo teaches a summary of the multimedia is a selected permutation of the content 
entities according to the associated ranks (col9, lines 19-42). 
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Response to Arguments 

4. Applicants' arguments filed 09/21/2006 have been fully considered but are not 
persuasive, 

a. Applicant argues that Yeo never orders different shots in a related sequence of 
shot using a DAG to provide a high-level interpretation of what is going on at a 
semantic level [Remarks, 7]. 

In response, the Examiner respectfully disagrees. Yeo does teach the use of 
directed graphs to order video segments (e.g,, combines visual and temporal 
information to capture the important relations within a scene and between scenes 
in a video,.. Video shots are first identified and a collection of key frames is used 
to represent each video segment.. .A platform is built on which the video is 
presented as directed graphs to the user, with each category of video shots 
represented by a node and each edge denoting a temporal relationship between 
categories) [see the Abstract]. 

b. Applicant further argues that Yeo does not describe the comparison of ordered 
content entities in a plurality of the directed acyclic graphs to determine a 
similarity of interpretations of the multimedia content [Remarks, page 8]. 
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In response, the Examiner respectfully submits that Applicant's arguments are 
substantially directed to the amended subject matter. The amended subject matter 
is addressed above with respect to the discussion of independent claim 1 . 

Conclusion 



5. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 



Norton et al. U.S. Patent No. 5,790,416 

Qian et al. U.S. Patent No. 5,838,830 

Harel U.S. Patent No. 5,873,081 

Qian et al. U.S. Patent No. 6,002,803 

Eleftheriadis et al. US. Patent No. 6,079,566 

Sezan et al. U.S. Patent No. 6,236,395 



Lermon et al. 



U.S. Patent No. 6,516,090 



issued: Aug. 4, 1998 
issued: Nov. 17, 1998 
issued: Feb. 16, 1999 
issued: Dec. 14, 1999 
issued: Jun. 27, 2000 
issued: May 22, 2001 
issued: Feb. 4, 2003 



Y. Wang et al., ^'Multimedia Content Classification Using Motion and Audio 
Information," IEEE, June 9-12,1997, Hong Kong, pp. 1488-1491. 
T. Zhang et al., '''^Hierarchical Classification of Audio Data for Archiving and 
Retrieving" IEEE, March 15-19, 1999, pp. 3001-3004. 



Application/Control Number: 09/518,937 Page 9 

Art Unit: 2176 

• R. Lienhart et al., ''Scene Determination Based on Video and Audio Features,'' 
IEEE, June 7-1 1, 1999, pp. 685-690. 

• E. Wold et al., ''Content-Based Classification, Search, and Retrieval of Audio,'' 
IEEE, Fall 1999, pp. 27-36. 

• J.M. Gauch et aL, "Real Time Video Scene Detection and Classification," 
Information Processing and Management, May 1999, pp. 381-400. 



6. THIS ACTION IS MADE FINAL, Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS firom the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the 
advisory action. In no event, however, will the statutory period for reply expire later than 
SIX MONTHS fi-om the mailing date of this final action. 
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Contact information 



7. Any inquiry concerning this communication or earlier communications from the 

examiner should be directed to Maikhanh Nguyen whose telephone number is (571) 272- 
4093. The examiner can normally be reached on Monday - Friday from 9:00am - 5:30 
pm. If attempts to reach the examiner by telephone are unsuccessfiil, the examiner's 
supervisor, Heather Hemdon can be reached at (571) 272-4136. 



The fax phone number for the organization where this application or proceeding is 
assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent Application 
Information Retrieval (PAIR) system. Status information for published applications may be 
obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Any response to this action should be mailed to: 

Commissioner for patents 
POBox 1450 

Alexandria, VA 22313-1450 



Maikhanh Nguyen 
Examiner, AU 21 76 
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